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Abstract 

The dissemination of good practices in MLE requires a more rigorous definition of their characteristics, including criteria and indica¬ 
tors to measure their quality. This article - after reflecting on the meaning and function of identifying good MLE practices - presents a 
systematic framework of 35 indicators for good MLE activities in Italian elementary schools. These indicators are grouped according to 
five general criteria of quality: adequacy of the teaching methods; competence and involvement of the actors; structuring and coherence 
of the activity’s organization; awareness of the underlying MLE theories as well as their appropriateness; and originality of the project. 
The framework emerged from a four-level process, which is also presented in the article: Based on an initial review of features attributed 
to quality MLE in scientific and institutional literature, a preliminary draft of criteria was elaborated and submitted to scholars and ex¬ 
perienced teachers, whose comments were sought in semi-structured interviews. This input fed into the final version of the framework, 
which may serve to support teacher trainings as well as the monitoring and (self-)evaluation of teachers’ MLE work. 
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Quality is not a thing. It is an event. [...] 
It is the event at which the subject becomes aware of the object. 

(Pirsig 1999, 239) 

In the field of media literacy education (MLE), 
as well as in other areas of schooling and life-long 
learning, there has been much talk about “good 
practices” for about twenty years now. Usually, these are 
referred to with the explicit aim to improve the quality 
of educational processes by highlighting experiences 
that deserve to be taken as a model by practitioners. 
This trend can be observed, for example, at teacher 
training seminars or academic conferences as well as 
in a number of journals, which - like this one - devote 
entire sections to the presentation of “voices from the 
field” that distinguish themselves by originality and 
innovation. 

The subj ect of goodpractice, then, in combination 
with the issue of educational quality, has become a 
common topic in political discourse, and pedagogy has 
often drawn on it uncritically. On the other hand, the 
question of good practice is also linked to the evaluation 
of MLE, which maybe is not a flagship of our field, 
being the subject of only few pioneering investigations, 
which - above all - have aimed at defining and testing 


learning outcomes to be expected in students at the end 
of MLE units (the components of so-called “media 
competence,” expressed in terms of knowledge, skills 
and habits) (Scrimshaw 1992; Worsnop 1996; Christ 
1997; Hobbs and Frost 2003; Aglieri 2005; Parola and 
Trinchero 2006; Hobbs 2007; Rivoltella 2008; Martens 
2010; Calvani, Fini and Ranieri 2011). The issue of 
evaluating the quality of MLE projects - understood as 
complex processes implemented by educators within 
an organization, entailing specific constraints, a budget 
and certain goals - remained in the shadow. 1 This has 
led to the paradoxical situation that, in the field of 
MLE, we talk about best practices, but often lack the 
criteria or precise benchmarks that would enable us to 
say which practices are actually good, and why. Many 
times, we simply rely on our gut feeling - which I 
personally trust a lot - but with the risk of falling into 
a dangerous subjectivism, which may lead us to over- 
or underestimate the educational experiences that we 

1 Note that, in English, the two words evaluation and assessment 
refer to two separate concepts, while in the Romance languages 
both learning assessment and educational process evaluation are 
referred to by the same term - evaluation in French, valutazione 
in Italian, evaluacion in Spanish, to mention just three examples. 
This, in turn, also results in a less rigorous conceptional difference 
in these languages. 
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witness or have come to know. 

In this paper, thus, I would like to address the 
issue of quality in MLE, with the aim to outline what 
is good practice and how we can identify it; moreover, 
I will propose a systematic framework of criteria and 
indicators for assessing the quality of MLE in the 
specific context of Italian elementary schools. 

1. Revising the concept of good practice in MLE 

In a penetrating and, in some respects, very 
ironic article, which appeared a few years ago in the 
British Educational Research Journal, Cofifield and 
Edward (2009) critically reflect on the widespread use of 
the concept of good practice in the jargon of pedagogy, 
politics and economics of education, especially in the 
area of vocational training. They highlight how, within 
a few years, the term “good practice” was replaced by 
“best practice” and then “excellent practice”, which 
leads the authors to wonder: “what next? perfect 
practice?”. Still not satisfied, British policy makers 
subsequently went on to advocate the ideal of “uniform 
excellence”, which refers to the goal of having first- 
rate practice for all, in every educational center of the 
country. While, from an ideal point of view, such an 
objective is certainly understandable, it is realistically 
impossible to achieve through the dissemination of 
standardized procedures, given the fact alone that every 
teaching situation is different; uniform excellence in 
education, therefore, would require robots instead of 
teachers and students in flesh and blood. 

Reasoning on the quality of MLE and the use 
we can make of good practices entails, in my opinion, 
finding answers to three interconnected questions: 1) 
who determines which practices are good; 2) according 
to which criteria; and 3) what is the function of 
identifying good practices in a field like ours. 

The neo-positivist assumption that it is possible 
to indisputably determine the quality of a process by 
simply referring to absolute standards is false. As 
recognized by the classical philosophical tradition, 
from Aristotle’s De Anima onwards, any evaluation is 
mainly based on a judgment, i.e., the action by which 
someone says something about something. Evaluation 
and assessment, in other words, are human, personal, 
subjective or inter-subjective acts based on criteria that 
are accepted by the reference group; evaluation cannot 
be thought of as the result of an impersonal or automatic 
process. Thus, it is crucial to ask by whom MLE practices 
should be evaluated to determine which of them are 


good and which are not. Obviously, to avoid the risk 
of falling into complete arbitrariness, such a judgment 
cannot be left to a single individual; on the contrary, it 
requires the consideration and integration of a plurality 
of points of view. Scholars, in particular those from 
academia, have always been considered as “experts” in 
the different fields of knowledge, their expertise being 
more of a theoretical and abstract nature. According to 
the tenets of the so-called “new educational research” 
(Wittrock 1986; Tochon 2000; Houssaye et al. 2002; 
Damiano 2006), however, we should not forget that 
knowledge of practice also resides in practitioners 
and that our understanding of education is best when 
theoretical knowledge and practical experience meet. 
This is even truer when it comes to good practices, 
which always need to be seen in their respective 
contexts (Alexander 1997). It is impossible, in other 
words, to say what is good and what is not in absolute 
terms, and this is the reason why, according to Coffield 
and Edward, we prefer the term good practice to that 
of best practice, since the second “also implies that 
there is only one approach which, if used, will solve 
any difficulties” (Coffield and Edward 2009, 375). As 
a consequence, direct knowledge of the real contexts in 
which MLE projects take place is essential in order to 
determine quality characteristics. 

To make comprehensible and acceptable value 
judgments that lie behind good practices, it is necessary, 
however, that the criteria they are based on are, first 
of all, explicit. Then, to decide which MLE practices 
are good and which are not, we cannot solely rely on 
economistic criteria of effectiveness (what makes 
students learn more?): firstly because effectiveness is 
not an absolute value, but related to a certain context; 
secondly, and mostly, because we may consider other 
values as equally or more important, such as equality 
of opportunity, or participation of all students in class. 
Therefore, we need to recognize that the quality of an 
MLE experience - just like the quality of all social 
actions in general (Pirsig 1999) - is shaped by a 
whole range of factors that cannot always be precisely 
quantified, but nevertheless must not be neglected: 
teachers’ ability, organization of time and space, 
teaching methods, involvement of external experts, 
availability of technical facilities, and so on. 

Finally, it is important to ask to what purpose 
we need to define good practices in MLE. Beyond the 
mere evaluative use of this construct - for instance, 
in the context of awards or to determine whom to 
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invite to a conference or which article to publish in 
a professional journal - today’s insistence on good 
practice, as mentioned before, is above all due to the 
tendency of policy makers to support policies outlining 
future developments of a certain sector based on 
examples of good practice that can serve as a model for 
practitioners. According to this logic, good practices 
are a tool for practitioners to improve the quality of 
certain educational services. 

It is primarily from this angle that the tool 
presented in the following pages has been conceived. 
Its aim is not to rank projects or teachers (sometimes 
a brief, intelligently designed MLE unit taught in a 
small school with few resources can be as admirable 
as one done in a more privileged context involving 
many material and human resources) nor to provide a 
lengthy list of activities to be implemented passively 
and uncritically (thus putting additional weight on the 
shoulders of already over-taxed teachers). Instead, 
the key objective is to provide an instrument for the 
development of educators’ competence in teaching 
MLE, geared towards implementing activities of 
“feasible quality.” Thus, this is a tool of self-evaluation, 
especially useful for questioning and examining our 
own experience, either individually or in a community 
of practice (Wenger 1998). 

This self-evaluative aim of the tool is the main 
difference distinguishing it from other approaches to 
the evaluation of good practices in MLE, such as the 
model put forward by Parola and Ranieri (2010). 2 
Within the context of an international project on MLE 
practices in six European countries, they evaluated 309 
experiences in different educational settings, according 
to a theoretical framework very similar to the one I am 
proposing (i.e., paying attention to diverse contexts and 
focusing on multiple aspects of educational processes) 
and based on the following indicators: “educational 
relevance, teaching approach, media use, sustainability 
of the experience, product, documentation quality, 
evaluation system and tools” (ibid., 91-92). According 
to the general objectives outlined by Parola and 
Ranieri, their evaluation model is meant to serve the 
testing of MLE programs rather than to be used for 
(self-)evaluation. This is reflected by some of the 
indicators and criteria, which cannot be applied by 
the teachers whose experience is being evaluated. 
For assessing the “evaluation system and tools”, for 

2 A comprehensive review of evaluation and assessment models in 
the area of MLE has recently been conducted by Martens (2010). 


instance, the criterion “appropriateness of assessment 
tools” (ibid., 98) is proposed, “appropriateness” being 
a rather subjective category for self-evaluation. Also, 
the teaching approach is described by only four general 
indicators (“planning appropriateness; effectiveness 
of methods; content accuracy and appropriateness to 
the student level; and student engagement”; ibid., 96). 
These may be useful for external evaluations, which 
tend to be based on written documentation, but they do 
not help teachers in improving self-awareness of their 
teaching methods. 

Moreover, while I consider the model by Parola 
and Ranieri a good example of MLE practice evaluation, 
it cannot avoid the logical short circuit I described 
before: the authors make the evaluation criteria very 
explicit, but do not explain how and why they decided 
that these criteria are relevant to understand whether a 
practice is good or not. 

2. Features of MLE good practice in international 
literature 

At international level, there are a number of 
indications and guidelines suggesting characteristics 
of what may be considered quality MLE. These 
can be found in official resolutions of international 
organizations, documents prepared by research 
institutions or professional associations, and overviews 
compiled by eminent scholars in the field (UNESCO 
1982;Aufderheide 1997;Masterman 1997; Buckingham 
2003; European Centre for Media Literacy 2005; French 
National Commission for UNESCO 2007; European 
Association for Viewers Interests 2008; The Aspen 
Institute Communications and Society Program 2010; 
Bonomi Castelli 2006; National Association for Media 
Literacy Education n.d.). In a previous paper, I have 
already given a detailed analysis of these texts with 
a view to comparing their recommendations (Felini 
2010a). Here, I will limit myself to summarizing some 
of the conclusions that guided the subsequent research 
path. 

First of all, the analysis of the above documents 
confirms the aforementioned lack of appropriate tools 
for evaluating MLE projects. In fact, the European 
Association for Viewers Interests (2008), the French 
National Commission for UNESCO (2007) as well 
as the European Centre for Media Literacy (2005) all 
call for the elaboration of evaluation systems at central 
and/or local level. Moreover, all of the texts consider 
MLE as key in the education of new generations, not 
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as a marginal exercise (to be relegated to some limited 
contexts or target groups) or an elective activity (to 
be chosen only by those who are interested). In the 
documents, also the idea prevails that media education 
can be better implemented in contexts of cooperation 
among people with different competence areas, with the 
external collaboration of media professionals and the 
involvement of families, as well as in interdisciplinary 
or cross-curricular units (Mastennan 1997; Buckingham 
2003; The Aspen Institute Communications and 
Society Program 2010; National Association for Media 
Literacy Education n.d., 2.4). Finally, another set of 
recommendations that recurs insistently concerns 
the use of active, participatory and discovery-based 
teaching methods, such as group discussions or the 
creation of media products (videos, posters, journalistic 
work etc.) (Masterman 1997; European Centre for 
Media Literacy 2005; Ceretti, Felini and Giannatelli 
2006). The NAMLE’s Core principles of media literacy 
education, in particular, highlight the importance of 
teaching methodologies, stressing that “HOW we teach 
matters as much as WHAT we teach”, and that “MLE is 
most effective when used with co-learning pedagogies” 
(National Association for Media Literacy Education 
n.d., 4.8, 3.4). 

Three issues, however, seem to emerge from 
the comparison of the examined papers and resolutions. 
Firstly, the highly abstract and general level of these 
texts leads to suggestions that are very far from school 
reality (UNESCO 1982; French National Commission 
for UNESCO 2007; European Association for Viewers 
Interests 2008). Obviously, documents issued by 
international organizations give priority to policy 
guidelines and related recommendations (teacher 
training, elaboration of national curricula, creation 
of networks, research development etc.), but the 
gap between what is outlined in these texts and the 
concreteness of everyday MLE is, in my view, a clear 
sign of the inability to understand that the quality of the 
latter is equally, if not more, affected from the bottom 
up: by the choices of individual schools, the actual 
availability of small equipment in each classroom, 
the relevance attached to MLE issues by the school 
community, educators’ competence, and families’ 
requests, to mention just a few. 

Secondly, the definitions and theories of MLE in 
several documents seem superficial or even trite, often 
marked by stereotyped catchwords. “Language skills”, 
“critical thinking”, “digital citizenship” or “literacy” 


are expressions that keep coming up, but seem to 
betray a lack of awareness that, behind these formulas, 
there are diverse pedagogies of MLE and different 
conceptions of media. In other words, my criticism is 
that at times these texts lack in-depth reflection on what 
MLE is, what is its purpose, and what is the connection 
between teaching goals and methods. This apparent 
lack of awareness and reflection, then, seriously blurs 
the clarity of the results one would need if the ultimate 
objective is indeed to improve the quality of MLE 
practices. 

Thirdly, the literature review on quality MLE 
gives us - besides individual indications (all valid, but 
in need of systematization) - one useful basic insight: 
that quality cannot be reduced to any single aspect. In 
fact, the documents deal with, at least, four aspects of 
quality to be considered in MLE projects: 

• the quality of the policies to support, develop and 
disseminate MLE; 

• the quality of the individuals and/or institutions 
providing MLE; 

• the quality of the organizational and infrastructural 
aspects of MLE activities; 

• the quality of the theoretical MLE background 
(i.e., both the underlying pedagogy as well as the 
conception of media and their role in society). 

3. The empirical research 

3.1 Objectives, research plan, and methodology 

Starting from these preliminary results, I 
developed an integrated system of criteria to identify 
quality MLE in Italian elementary schools. 3 The 
specification of the educational and geographical 
context (elementary schools in Italy) has been made 
necessary to respect the principle of contextualization 
of good practices. In line with the argument presented 
in Section 1, the focus of the research was to identify 
quality criteria, but also to address the issue of who 
defines which criteria are characteristic of good practices 
in MLE. In other words, if assigning the label “good” to 
a certain practice is related to making a value judgment, 

3 According to Italian laws, MLE is not mandatory in all schools, 
even though the Indicazioni nazionali (National Curriculum Stan¬ 
dards) currently in force provide a number of learning outcomes 
that could be easily included in this area (Felini 2010b). In elemen¬ 
tary schools, MLE is currently taught on a voluntary basis by mo¬ 
tivated teachers, generally as part of interdisciplinary courses or in 
workshops offered in the afternoon to enrich the basic curriculum; 
sometimes, these are co-taught by two teachers or a teacher and a 
technical instructor. 
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the underlying problem is to understand beforehand 
who, and on the basis of which criteria, evaluates that 
practice as it has been designed and conducted. 

Therefore, the research plan used to develop a 
framework of quality standards for MLE practices in 
Italian elementary schools consisted of four phases: 

1. scientific and institutional literature, both Italian and 
international (see Section 2), was analyzed with a view 
to identify recurring criteria and indicators by which 
good practices of MLE are defined. These criteria were 
grouped and integrated into a preliminary draft of an 
organic framework; 

2. this first list of indicators was submitted to three Italian 
MLE scholars, whose comments and recommendations 
regarding the draft framework were recorded. They 
were selected because of their expertise in the field, 
confirmed by their publications, participation in 
research groups, and their work as consultants for 
national or regional boards of education. Obviously, 
these scholars’ opinions are not free of subjectivity, but 
they are nonetheless informed by their knowledge of 
recent educational research done in MLE; 

3. the same list of indicators was submitted to 
five elementary school teachers with significant 
professional experience in MLE. All of them have 
worked as elementary teachers for at least twelve years 
and as MLE teachers for at least five. In addition, four 
of them have experience as trainers of other educators 
in the field of MLE. Their comments (which were also 
recorded) have been particularly valuable in order to 
bridge the gap between theory and the everyday school 
reality mentioned before; 

4. the synthesis of the information gained from 
the literature review and the suggestions gathered 
during the expert interviews were incorporated into 
a comprehensive proposal of quality criteria, which 
is presented below (see Section 4). This work of 
systematization is also based on both my theoretical 
knowledge and field experience of nearly fifteen 
years, including the co-ordination of the design and 
development of a MLE curriculum for elementary 
schools (Ceretti, Felini and Giannatelli 2006). 

As one can see, the entire research path is 
based on a critical, constructivist and inter-subjective 
epistemological paradigm (Habermas 1985; Gadamer 


1986; Caronia 1997; Mortari 2007), which rejects the 
notion of an objective view that exists regardless of the 
subject; this is particularly crucial in this case, where 
setting criteria as absolute values could hide either an 
insufficient theoretical elaboration or an ideological 
vision. Moreover, since quality criteria potentially 
affect the choices and practices of a large number of 
individuals, they need to be shared through a transparent 
process in which the personal positions of everybody 
involved are made as explicit as possible, and which is 
open to scrutiny and criticism. Under these conditions, 
it should be possible to build a framework based on the 
synthesis of a large range of views. The aim of this is 
not to establish a true list of quality criteria, but to create 
something that is reliable because it has been given 
value through a methodical and documented process. 

The academic experts consulted during the 
research were: 

• Alberto Agosti, Professor of Education at the 
University of Verona; 4 

• Floriana Falcinelli, Professor of Education at the 
University of Perugia; 5 

• Roberto Fame, Professor of Education at the 
University of Bologna. 6 

They participated in semi-structured interviews 
(Fontana and Frey 1994) conducted between September 
and December 2011 (see protocol in Annex 1); during 
the interviews, respondents were given sufficient time 
to carefully read the draft list of criteria. The audio files 
containing the recorded interviews as well as the draft 
list of indicators that the interviewees were shown can 
be found at: http://qualityMLE.wordpress.com (with 
the permission of the respondents). 

Using the same method (see protocol in Annex 
2), in July 2012 I interviewed five elementary school 
teachers - selected according to the criteria outlined 
above -, and presented them, too, the draft framework of 
quality criteria and indicators. The teachers interviewed 

4 Agosti has worked on film education for more than 20 years and 
published books as well as several academic articles in this field. 

5 Falcinelli has published books and articles on educational media 
and MLE for more than 20 years. She coordinated a large research 
project on MLE in elementary schools in the region of Umbria 
(funded by the regional Office of Education) as well as a national 
research project on teaching practices involving media (funded by 
the Ministry of Education). 

6 Fame has published books and articles on image, film, and televi¬ 
sion education for more than 30 years. He coordinated a national 
survey on MLE in Italian elementary and middle schools (funded 
by the regional Communication Commissions). 
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were: 

• Susanna Lemmi, “San Rocco” Primary School, 
Portoferraio (Livorno); 

• Paola Mastronardi, “Giovanni Pascoli” 
Comprehensive School, Quattro Castella (Reggio 
Emilia); 

• Liana Peria, “Giuseppe Giusti” Comprehensive 
School, Campo nell’Elba (Livorno); 

• Emanuela Pucci, “Gianni Rodari” Primary School, 
Nettuno (Rome); 

• Manuela Valenti, “Arturo Toscanini” 
Comprehensive School, Parma. 

The audio files containing the recordings of these 
interviews can also be found at: http://qualityMLE. 
wordpress.com. 

The interviews were examined carefully 
using qualitative content analysis and, specifically, 
the summary technique (Mayring 1988, 53), which 
consists in the reduction of the material “in such a way 
as to preserve the essential content and by abstraction 
to create a manageable corpus which still reflects the 
original material” (Titscher et al. 2000, 62). By this 
approach, the interviewees’ input was summarized, 
concept by concept, in bulleted lists. Recurring ideas 
were underlined in different colors, as were original 
thoughts, hesitant and confident answers, doubts and 
questions raised. In this way a long series of elements 
emerged, that formed the critical basis for revising and 
elaborating the initial draft of indicators. 

3.2 Findings 

While the recordings used for the qualitative 
content analysis can be accessed online, I would like to 
summarize here the main ideas and conclusions drawn 
from the interviews, substantiated by some meaningful 
excerpts. It is important to remember, though, that the 
aim of my research has not primarily been to collect 
general ideas about good practices in MLE - which 
nonetheless emerged and were recorded -, but to 
review, amend and improve my draft list of criteria and 
indicators. Therefore, the findings outlined here are 
those prior to the elaboration of the final list, which is 
presented in the next section. 

Feedback from the scholar interviews led me 
to increase the number of quality indicators and revise 
their order. In particular, criteria regarding parental 
involvement, teacher training and the quality of the 
media products created during an MLE unit were added 
to the framework. 


Moreover, Professor Agosti highlighted the 
importance of two ideas: 1) the need of using active 
teaching methods, which allow children to express and 
compare their ideas as well as to work individually; 2) 
the difficulty in applying the concept of good practice 
and the necessity to take into account, in a research 
project like this, the sensitivity and actual possibilities 
of teachers. As Agosti pointed out: 

Placing the word “good” before “practice” 
constitutes a pre-judgment, which is why we [as 
scholars] put ourselves in a dangerous position. 
What does it trigger at the emotional level to 
be asked “what is a good practice ”, especially 
in a practitioner? One cannot immediately put 
oneself in the role of a judge; especially, 
one cannot ask a practitioner to do so, if it’s 
you, an academic, to ask the question. The 
risk is that what practitioners really want to say 
will never emerge, mainly because they want 
to look good 7 (interview with Alberto Agosti, 
Dec. 9, 2011). 

ProfessorFalcinelli, on the other hand, elaborated 
on the “cycle” of good practices (adequate planning 
—► careful execution —> constant documentation —> 
reflection on the experienced practice) as well as on 
the overlap between the objectives of MLE and those 
of other disciplines: “In addition to the specific aims 
of MLE,” she said, “I would also identify intersections 
with other competence areas, both cross-disciplinary 
and in terms of citizenship” 7 8 (interview with Floriana 
Falcinelli, Sept. 30, 2011). Moreover, she stressed 
that children should be given the opportunity to use 
technological tools themselves during activities. In her 
view, “it’s quality if children use the media themselves, 
not the teacher. The children should be the protagonists 
using the tools” 9 (ibid.). 

Prof. Fame, finally, focused on the meaning of 

7 “II fatto di anticipare la parola ‘buone’ davanti a ‘pratidle’ costi- 
tuisce una pregiudiziale, perche ci si pone in una posizione peri- 
colosa. Cosa pud suscitare a livello emotivo, specialmente a un 
pratico, il sentirsi chiedere cos’e una buona pratica? Non ci si pud 
porre immediatamente in una posizione di giudizio, ne soprattutto 
chiedere a un pratico di farlo, se sei tu, accademico, a porre la 
questione. II rischio e che non emerga cio che realmente i pratici 
vogliono dire, soprattutto perche vogliono fare bella figura.” 

8 “Oltre alle finalita specifiche della media education, individuerei 
anche le intersezioni con altre competenze, disciplinari e di cit- 
tadinanza.” 

9 “E qualita se i bambini usano i media, non l’insegnante. I 
bambini devono essere protagonisti nell’uso degli strumenti.” 
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good practices, stressing that their elaboration should 
lead to new knowledge about teaching. He also insisted 
that good MLE should contribute to the development 
of critical thinking skills, alongside objectives such as 
fostering relational and communication competencies 
as well as self-confidence. As he elaborated: 

If I should think ofan element that determines the 
quality of MLE in elemen tary schools, I would 
say it’s the ability of the experience to attain the 
acquisition of appropriate critical thinking skills 
in children (i. e., the ability to make comparisons, 
observations, linguistic operations, evaluations 
etc.). These are things that nobody teaches but 
school. Schools must include media in their 
activities, thinking of them as texts, with the 
same dignity as a poem. However, they must 
choose good things, things that have value. 
It’s not true that bad movies work. This is the 
teachers ’ responsibility, a huge responsibility 10 
(interview with Roberto Fame, Nov. 3, 2011). 

Unlike the feedback provided by the academic 
experts, the teachers’ input was far more homogeneous. 
All of them stressed the following points: 

• The methodological aspects of good MLE, in 
particular, active teaching, effective use of group 
work, and children’s active involvement. Manuela 
Valenti, for instance, said: “I’d focus more on the 
organization within the classroom: MLE cannot 
be taught as a lecture. Teachers should work more 
with groups and in workshop settings, with the 
aim of fostering skills [instead of transmitting 
mere knowledge]. [What is also important is] the 
capacity to handle the class in different ways, each 
specifically chosen for a certain situation and the 
particular type of learning that is to be achieved” * 11 

10 “Se devo pensare a un elemento che faccia la qualita della media 
education nella scuola primaria, direi la capacita dell’esperienza di 
raggiungere l’acquisizione di adeguate capacita critiche nei bam¬ 
bini (ovvero, la capacita di fare confronti, osservazioni, operazioni 
linguistic he, valutazioni...). Queste sono cose che se non fa la 
scuola non fa nessun altro. La scuola deve portare i media dentro 
le sue attivita, considerandoli come dei testi, con la stessa dignita 
di una poesia: deve pero scegliere le cose belle, le cose che hanno 
valore. Non e vero che i him brutti servono... E questa la respon- 
sabilita degli insegnanti, una responsabilita enorme.” 

11 “Io punterei di piu sull’organizzazione all’interno della classe: 
la media education non pud essere una lezione frontale, bisogna 
lavorare di piu per gruppi e in laboratorio, e per competenze. La 
capacita di gestire la classe in diversi modi, ciascuno specihca- 


(interview, July 3, 2012). Liana Peria, speaking 
of her past experiences, also asked herself: “What 
was good about those experiences? First of all... 
the newness... the children’s involvement, and the 
fact that they got to produce something concrete” 12 
(interview, July 18, 2012). 

• The need to adequately document activities so 
as to ensure reproducibility. One teacher said, 
for instance: “I would insist on documentation, 
because in this way, other colleagues can see what 
has already been done” 13 (interview with Emanuela 
Pucci, July 13, 2012). And another one explained: 
“We always see documentation as a burden; we don’t 
have a project-oriented culture. But it is necessary 
and helpful for our work... [Documentation] is not 
only a formal goal; in fact, it is essential to make 
the process reproducible and constitutes a wealth of 
expertise for the whole institute. Otherwise, MLE 
will not become something one would normally 
do, but something that is carried on by only a few 
motivated teachers. The aim, however, is to make 
MLE everyday” 14 (interview with Manuela Valenti, 
July 3, 2012). 

• The effectiveness of co-teaching, which is becoming 
increasingly rare nowadays. 15 “Today we have the 
problem that there’s only one teacher at a time [in 
the classroom], since co-teaching has come to an 
end. This is an obstacle to involving the children one 
by one, because sometimes we can’t do much more 

mente scelto per quel momento e per quel tipo di apprendimento 
che si vuole fare raggiungere.” 

12 “Cosa avevano di buono queste esperienze? Prima di tutto... la 
no vita... il coinvolgimento della classe e il fatto di arrivare a pro- 
durre qualcosa di tangibile.” 

13 “Insisterei sulla documentazione, perche cost altri colleghi pos- 
sono vedere cosa gia e stato fatto.” 

14 “La documentazione la vediamo sempre come un peso, non 
abbiamo la cultura del progetto. Ma e necessaria e funzionale al 
nostro lavoro... [Documentare] non e pero un obiettivo solo for- 
male, perche si tratta di qualcosa di indispensabile per rendere il 
percorso riproducibile e un bagaglio di competenza per tutta la 
scuola. Altrimenti la media education non diventa qualcosa che 
normalmente si fa, ma resta qualcosa che solo alcuni insegnanti 
motivati portano avanti. Lo scopo, invece, e trasformare la media 
education in quotidianita.” 

15 The Italian elementary school lasts five years, and each teacher 
follows a group of children over the entire period from grade 1 
to 5. Between 1985 and 2008, teachers predominantly worked in 
pairs in two parallel classes; they taught all the main subjects, and 
there was the possibility to co-teach in the classroom for some 
hours a week. This possibility is very effective from an academic 
point of view, but, obviously, it is also very costly; thus, it has been 
severely limited by the latest educational reforms (Law 169/2008). 
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than lecture” 16 (interview with Paola Mastronardi, 
July 3, 2012). 

In line with this, the interviewees were also asked 
to elaborate on what else they considered obstacles to 
teaching MLE at elementary school level, which was 
necessary to avoid the inclusion of indicators referring 
to an “unattainable quality”. It turned out that, apart 
from the already noted scarcity of co-teaching, the main 
obstacles are related to teachers’ resistance and lack of 
motivation and training, on the one hand, and to limited 
time, space and technological equipment, on the other. 
As Susanna Lemmi pointed out: “First of all, there’s 
a problem regarding our training, as teachers; then, 
there’s a financial problem, affecting the acquisition of 
equipment” 17 (interview, July 13, 2012). 

Finally, all teachers interviewed expressed 
strong doubts about considering the institutionalization 
of MLE at school level, which would entail the inclusion 
of MLE into the POF 18 as a quality criterion. As one 
teacher explained: 

I would put this idea of institutionalizing MLE 
in brackets. It’s already a daily battle in the 
School Committees, and certain things just go 
on by inertia. On the one hand, the inclusion of 
MLE in the POF is very hard to achieve and, 
on the other, it does not even guarantee that 
things will actually be done. Of course, it would 
be desirable to include MLE in the POF, but I 
don’t see this as something indispensable 19 
(interview with Manuela Valenti, July 3, 2012). 

And another teacher confirmed: “I don’t kn ow how the 
institutionalization could be agreed on by all members 
of the School Teachers Committee - it’s not even 


16 Oggi c’e la difficolta di essere solo un insegnante alia volta, per 
colpa della fine delle compresenze: cio e di ostacolo al coinvolgere 
i bambini uno per uno, perche talvolta non si pud far molto altro 
che una lezione frontale.” 

17 “Anzitutto, c’e un problema di formazione nostra, dei docenti; 
poi c’e un problema economico, di reperimento degli strumenti.” 

18 POF (= Piano dell’Offerta Formativa) is a public document, an¬ 
nually updated, which contains the mission statement and the edu¬ 
cational activities offered in each Italian school institute. 

19 “Questa parte dell’istituzionalizzazione la metterei un po’ tra 
virgolette: e una battaglia quotidiana nei Collegi, certe cose vanno 
avanti per inerzia. II fatto che la media education venga messa 
nel POF, da un lato e molto difficile da ottenere e, dall’altro, non 
garantisce che poi le cose vengano fatte davvero. Chiaramente e 
auspicabile che la media education sia nel POF, ma non la vedo 
come cosa indispensabile.” 


necessary” 20 (interview with Paola Mastronardi, July 

3, 2012). As a result of these comments, the criteria 
relating to the institutionalization of MLE, on which the 
international literature and the scholars interviewed so 
strongly insisted, were removed from the final version 
of criteria to identify good practice in MLE. 

Overall, the teachers’ input led me to review or 
delete certain criteria and indicators, which appeared 
disproportionate compared to the actual possibilities at 
school, and to put more emphasis on others. Regarding 
the relationship with students’ families, for example, 
one teacher remarked: “This is very interesting: I’ve 
never really cared much about it, never even thought 
of this. But it’s true: it’s important” 21 (interview with 
Paola Mastronardi, July 3, 2012). As for the importance 
of originality as a criterion of MLE quality, I recorded 
different opinions. Most of the interviewees agreed that 
“of course, originality is important, not only in MLE, 
but in everything we do” 22 (interview with Susanna 
Lemmi, July 13, 2012), or that “it certainly constitutes 
an element of quality” 23 (interview with Emanuela 
Pucci, July 13,2012). Some practitioners, however, had 
“doubts about this originality thing. First of all, because 
we always start from familiar ground; only then one 
can introduce something more unusual and special” 24 
(interview with Manuela Valenti, July 3, 2012). As a 
result, the originality criterion remained part of the final 
framework, but was substantially revised. 

4. Criteria and indicators for quality MLE in Italian 
elementary schools 

As a result of the research outlined above - i.e., 
the analysis of scientific and institutional literature as 
well as the interviews with scholars and practitioners - 
I would like to present a systematic proposal of criteria 
and indicators for quality MLE in Italian elementary 
schools (see Table 1 in the Appendix). 

Formally, the features of good practice have 
been grouped according to five main criteria of quality 
(teaching meth ods, actors, organization, theoretical 

20 “L’istituzionalizzazione non so quanto possa essere condivisa 
dal Collegio Docenti: non e nemmeno indispensabile.” 

21 “II tema del rapporto con le famiglie e molto interessante, anche 
se io non l’ho mai curato molto; non ci ho mai pensato. Ma e vero, 
e importante.” 

22 “Certamente l’originalita e importante, non solo nella media 
education, ma in tutto.” 

23 “L’originalita e sicuramente un elemento della qualita.” 

24 “Ho dei dubbi su questa cosa dell’originalita. In primo luogo 
perche si parte sempre da un terreno conosciuto: solo dopo ci si 
pud lanciare in qualcosa di piu inconsueto e particolare.” 
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background, and originality), each of which is 
further refined by several sub-criteria (or areas of 
investigation) and corresponding sets of indicators. The 
terms “criteria” and “indicators” are used as proposed 
by Castoldi (1998); thus, “criteria” are the aspects of a 
given phenomenon (in this case MLE activities) which 
define its value. Criteria may be found in different 
“areas of investigation”. (The organization of a given 
educational activity, for instance, is a quality factor 
which we may consider in different areas: at school or 
class level, in terms of the use of equipment or human 
resources, with regard to the degree of interconnection 
with other schools or institutions in the area, etc.). 
Finally, for each criterion and area, the framework 
suggests sets of “indicators”, i.e., observable pieces of 
evidence defined in operational terms, which indicate if 
(and to what degree) the quality criteria are met. 

At the content level, on the other hand, the 
research process outlined before led me to propose five 
quality criteria, which can be applied to MLE activities 
in Italian elementary schools (see also Figure 1): 

a) the adequacy and effectiveness of the teaching 
methods; 

b) the competence and involvement of the actors in 
the MLE activity, as well as the forms of support 
provided by the school or other institutions 
participating in the project; 

c) the effectiveness, structuring and coherence of the 
project’s organization; 

d) the awareness of the underlying MLE theories as 
well as their appropriateness; 

e) the originality of the MLE activity. 


a) Teaching 
Methods 


Originality b)Actors 

Quality of MLE in 
Italian Elementary 
Schools 


d) 

Underlying c) 

MLE Organization 

Theories 

Figure 1 - Criteria of quality for MLE in Italian elementary schools. 


Considering each of these criteria and different 
areas of investigation, I developed and systematized 
35 quality indicators, on the value of which both the 
scholars and teachers interviewed had given their 
feedback. This system is presented in Table 1 (please 
see Appendix) and will be commented on further below. 

Regarding the first quality criterion (teaching 
methods), I have identified six indicators relating to 
the methods used in teaching MLE to children: all 
techniques encouraging students’ active participation, 
exploration, discovery, discussion, personal reflection 
and enrichment of their media experience were 
considered significant within the context of quality in 
MLE. 

Quality, for the most part, depends on people. 
This is why the second criterion (actors) refers to those 
involved in an MLE process: children, teachers, parents 
and media professionals. As for children, their direct 
participation as well as activities specifically designed 
for the group the teacher is working with (heeding, if 
necessary, but not slavishly following what guides or 
professional journals suggest) are considered crucial 
with regard to quality in MLE. Teachers’ training is 
another quality factor, which may include specific 
courses, but also participation in peer discussions. 
Parental involvement, then, is considered significant 
as well, in particular because children’s media use 
mostly takes place at home, and many parents are 
quite concerned about what their children may see on 
television or the internet. Finally, media professionals 
can provide MLE teachers with valuable input both 
in the preparation of and during the activity, thus 
enriching the teachers’ explanations with their practical 
knowledge and experience. 

Regarding the third criterion of quality 
(organization), three areas have been identified, each of 
them specified by a number of indicators: 

1. structuring of the MLE unit: including the existence 
of a written project plan, interdisciplinary links between 
MLE and other subjects (in line with class, school 
or national curricula), learning assessment as well as 
forms of (self-)evaluation regarding the activity, all of 
which are considered quality indicators; 

2. equipment: the students’ possibility to use the tools 
(cameras, computers etc.) themselves as well as the 
suitability of the equipment for the purposes of the 
respective activity are considered indicators of quality; 
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3. documentation and dissemination of results: the 
continuous documentation of all activities with the aim 
to increase media awareness and knowledge among 
families, schools, local communities and teachers 
through meetings, conferences, competitions or teacher 
trainings is considered an important quality indicator 
as well. 

Concerning the underlying theoretical 
background of MLE practices, two aspects were 
considered crucial quality factors: on the one hand, a 
clear awareness among all the teachers involved in the 
project about the expected outcomes for the children, 
both with regard to MLE objectives and objectives 
li nk ed to other disciplines as well as in terms of key- 
competencies qualification (relational skills, self- 
confidence, respect for equipment and workplace, 
accuracy, team working skills etc.); on the other hand, 
the existence of an explicit reference theory regarding 
the role of media in today’s society and the role of MLE 
in educating tomorrow’s citizens. These two elements 
were considered indicators of quality MLE, since they 
can help to avoid impromptu teaching. 

Finally - and although there were different 
opinions about this among the teachers I interviewed 
- a fifth MLE quality criterion was added: originality. 
This refers not only to the innovative character of the 
activity’s content, which needs to relate to the current, 
“liquid” (media) reality, but also to the originality of 
the teaching methods, to be adapted to contexts and 
addressees, as well as to the originality of the media 
message (video, journal, poster, photo book etc.) that is 
produced by the students during the unit. In this sense, 
a lack of originality has been considered symptomatic 
of a limitation to the most common and worn-out 
teaching routines, which can hardly respond to a media 
world in constant flux nor to the changing needs of new 
generations of students. 

Overall, compared to other frameworks 
analyzed in Sections 1 and 2, the methodology used 
to develop the framework outlined above differs 
substantially, in particular because it emerged at the 
crossing of the different points of view by the scholars 
and practitioners. In terms of content, however, many 
items in this framework can also be found in others - 
with two significant exceptions. 

Firstly, it does not contain criteria pertaining 
only to the political level, such as international 
cooperation policies or actions to mobilize stakeholders. 


These are considered relevant, for example, by the 
French National Commission for UNESCO (2007), 
the European Association for Viewers Interests (2008) 
and the European Center for Media Literacy (2005); 
in the present list of criteria, having a different goal 
and audience, such political aspects are excluded, as 
the framework is targeted towards elementary school 
teachers with the aim to help them improve their own 
MLE activities. 

Secondly, if we compare this framework to 
other documents developed to guide educators and 
based on pedagogical theories, there are some apparent 
differences as well, mostly because these texts are not 
expressly designed for elementary schools in one specific 
country. Moreover, they tend to be much less detailed. 
Masterman (1997), for example, lists only five general 
guidelines (concerning active teaching, school-family 
collaboration, involvement of media professionals, and 
cross-curricular design); and as observed in Section 
1, Parola and Ranieri’s model (2010) also has certain 
limitations. Bonomi Castelli (2006), on the other hand, 
presents a more comprehensive list of recommendations, 
but does not consider factors such as underlying 
theories of media and MLE, learning assessment, or 
originality. Finally, the Core Principles of the National 
Association for Media Literacy Education (n.d.), 
while designed for the specific context of the United 
States, also bear significance for educators in the rest 
of the world, since the principles are rooted in sound 
educational values, such as critical thinking, expanded 
literacy, participative citizenship, and life-long MLE. 
Many of the guidelines are very theoretical, however. 
They can help educators to clarify their conception of 
MLE - which is also a very important goal - but do not 
support them in evaluating the organizational aspects 
of their practical activities (school/family relationship, 
teacher training, course structuring, involvement of 
media professionals, originality, documentation and 
dissemination of results). 

For all these reasons, the list of quality criteria 
and indicators presented in this paper can be a specific 
tool for Italian teachers who want to improve their own 
MLE activities as well as for teacher trainers working 
in the field. 

5, Implications for practice and future research: 
The framework as a tool for teachers and teacher 
educators 

After ten years of experience in training 
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MLE teachers, I can say that, at least in Italy, this 
kind of training is frequently faced with a number of 
challenges. To begin with, there is always a spasmodic 
interest in technology on part of the teachers, as if 
teaching MLE automatically becomes easier when 
you leam how to edit a video or find a new tool on 
the web. The theoretical aspects are usually perceived 
as unnecessary chatter, and some training participants 
will not even accept methodological and didactical 
suggestions because they prefer to continue to teach 
in the way they have long grown accustomed to in 
their daily work. Mental rigidity, thus, is the greatest 
enemy of educators. In addition, training interventions 
are chronically constrained by limited time frames, in 
which the trainer would like to make the participants 
find out many things - too many, sometimes, compared 
to what can realistically be achieved. 

This is why the framework of quality indicators 
presented in this article might be used in such situations, 
since it has the advantage of providing a one-page only 
check list of indicators. This conciseness, however, is 
not bought at the cost of a broad perspective on MLE 
practices at school; on the contrary, as recognized by 
some of the teachers interviewed, 25 the framework 
allows them to consider a wide range of aspects: 
organizational issues; the relationships among students, 
teachers, parents, and professionals; teachers’ activities 
in the classroom as well as before and after class; 
the production of media messages by the children, 
the relationship between theory and practice, the 
links between MLE and other disciplines, and so on. 
Therefore, this grid of indicators lends itself especially 
to teachers who have already had at least one MLE 
experience and , thus, can do a self-evaluation of their 
25 Manuela Valenti, for instance, said that the framework is “very 
detailed and well-structured. It considers a wide range [of aspects] 
that allows you to look at projects from different angles: and this 
is vital” (“[la griglia, in generale, mi sembra] molto dettagliata e 
strutturata. E un ventaglio talmente ampio che da la possibility di 
guardare ai progetti da molti punti di vista: questo e fondamentale”) 
(interview, July 3, 2012). And as Susanna Lemmi commented, the 
framework “is definitely useful, but it can also frighten people, 
because everything that requires more work is frightening. How¬ 
ever, if a teacher wants to work seriously, this framework can be 
a useful tool. Some indicators can help during the initial planning 
phase, others in the course of the project, and others again during 
the evaluation” (“[la griglia] e senz’altro utile, ma puo spaventare, 
perche tutto cio che ci chiede di lavorare di piu spaventa. Ma se 
uno vuole lavorare, questo puo essere un buono strumento. Alcuni 
punti servirebbero prima, nella fase di progettazione, altri durante 
lo svolgimento del progetto e altri ancora nella fase di valutazio- 
ne”) (interview, July 13,2012). 


own work. This can be done either individually or 
within a group of peers, as the framework can be used as 
an effective tool to guide both (self-)reflective analysis 
and discussion, which again can trigger - and this is 
the function of good practices as patterns of action 
- a process of improving MLE by the practitioners 
themselves. 

In line with this thought, Castoldi (1998, 37) 
reminds us that the indicators are to be understood as 
“alarm devices, useful to indicate the proper functioning 
of an education system”. So, also in the case of the 
framework presented here, good practices are not 
necessarily those which exhibit all indicators listed, as 
the proposed tool mainly serves to provide evidence 
indicating, basically, what went well and constituted 
the value of the MLE activity. For this reason, the 
absence of some of the features outlined above does 
not necessarily indicate a negative experience. 

The added value of this tool, moreover, is that it 
signals the strengths and weaknesses of a MLE project 
not based on just one individual’s judgment, but-within 
a paradigm of inter-subjectivity - on the basis of the 
results gained through four levels of shared elaboration: 
the scientific literature that served as a basis for the first 
draft of criteria, the comments made by the scholars, the 
input based on the teachers’ practical experience, and 
my work of synthesis and systematization. The quality 
criteria presented here, thus, have value because they 
were given value, in a methodical, documented process. 

The next step in the near future is to test the 
educational effectiveness of this framework in the 
context of actual MLE trainings for elementary 
school teachers. Aspects to be looked at in particular 
will include: the clarity of the indicators as they are 
formulated, the amount of information necessary to 
apply the grid to a project implemented by others, and 
the grid’s potential to lead to a more critical analysis 
of MLE experiences that goes beyond the usual 
conglomeration of mere impressions on children’s 
satisfaction and participation. In fact, only a greater 
awareness of what one does can guide teachers - and all 
of us - to continuously improve the practice of MLE. 
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Appendix 

Table 1 - Systematic framework of criteria and indicators of quality for MLE in Italian elementary schools. 


Criteria of 
quality 

Sub-criteria (areas 
of investigation) 

Indicators 

a) Teaching 
methods 

Methods and 
techniques used in 
working with 
students 

1. Inclusion of both media analysis (texts, language use, consumption behaviors etc.) and 
media production by students; 

2. use of both individual and group work; 

3. teacher’s stimulation of critical reflection on what students discover or produce in class; 

4. teacher’s stimulation of discussions among students, giving everybody the possibility to 
form and express their opinions; 

5. use of active and participatory methodologies, followed by reflection on the experience; 

6. inclusion of different media products than those the children normally consume: in par¬ 
ticular, products of relevant narrative, aesthetic, social, or historical quality. 

b) Actors 

Centrality of 
children 

7. Educational design based on teachers’ knowledge of children, their media experience 
and consumption as well as their socio-cultural context; 

8. children’s active and personal participation in the various activities. 

Teachers’ training 

9. Assessment of teachers’ competence before their assignment to MLE courses/units; 

10. if needed, organization of specific teacher training courses before launch of a planned 
MLE project; 

11. exchange with other teachers involved in MLE activities, and peer-to-peer evaluation. 

School/family 

relationship 

12. Sharing with families the meaning of MLE for their children; 

13. creation of forms of parental involvement to achieve the educational outcomes; 

14. training for parents on family media management. 

Involvement of 
media professionals 

15. Participation of media professionals in the design of MLE projects; 

16. meetings of students with media professionals. 

c) Organization 

Structuring of the 
MLE course 

17. Existence of a written syllabus stating all relevant information regarding the planned 
MLE activity; 

18. interdisciplinary links between the MLE unit and students’ other educational activities; 

19. integration of the project within a school-wide MLE plan, and its possible inclusion into 
the POF; 

20. assessment of students’ learning; 

21. teachers’ self-evaluation of the course, based on the evidence collected and conducted 
individually or with colleagues. 

Use of equipment 

22. Availability of technological equipment appropriate for the aims of the project; 

23. use of equipment by the children themselves, as often as possible. 

Documentation and 
dissemination of 

results 

24. Continuous documentation of all activities (syllabus, diary of activities, videotaping of 
lessons, media products produced by the students etc.) and archival of these documents 
and all relevant information in a specific file/folder; 

25. presentation of results to the school and/or town community; 

26. participation in competitions; 

27. presentation on the activity at conferences and/or teacher trainings. 

d) Underlying 
MLE 

theories 

Awareness of the 
goals of the activity 

28. Clear formulation of the purpose and objectives to be achieved by students, both in 
terms of MLE outcomes, outcomes linked to other disciplines, and outcomes related to 
key skills qualifications. 

MLE concept 

29. Existence of an explicit concept of the media and their role in contemporary society; 

30. reference of the activity to explicit concepts of MLE and media competence. 

e) Originality 

Originality of the 
MLE activity and its 
contents 

31. Attention to current media reality and recent trends in the ever-changing media world. 

Originality of the 
media message 
produced by students 
during the activity 

32. Originality of the language chosen by the students or of the creative solutions adopted 
in the message; 

33. aesthetic value of the media message or of the creative solutions adopted in it; 

34. communicative effectiveness of the message in view of the target audience. 

Originality of the 
teaching methods 

35. Originality of the adopted teaching methods, especially in terms of their adaptation to the 
needs of the student group and its context. 
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Annex 1: 

Protocol for the interviews with scholars in the area of MLE at school 

Goals of the interview 

a) Verify the usefulness of the tool; 

b) verify the structure of the framework, both from a formal and content point of view; 

c) verify the significance of each part of the framework, in particular with regard to the criteria and sub¬ 
criteria; 

d) verify the significance of the indicators identified and detennine the possible lack of other relevant indicators. 

Script for the interviewer 

Part 1 (before showing the draft) 26 

1. In your opinion, how can teachers and schools improve the quality of MLE projects? What is needed to 
achieve this goal? 

2. Can a frame of indicators be useful? What could be the added value of such a framework? 

3. In your opinion, what are the elements that contribute to the quality of MLE in elementary schools? 

Part 2 (a fter showing and explaining the draft) 

1. What do you think about the five general areas that compose the framework? 

2. In your view, does it make sense to consider originality as a quality criterion for MLE projects? 

3. In your view, does it make sense to consider the institutionalization of MLE as a quality criterion for MLE 
projects? 

4. What concepts exist for evaluating the quality of individual activities carried out by the teacher during a MLE 
unit? 

5. Is evaluation appropriately considered by the framework? 

6. Do you think that the indicators identified for the first criterion (underlying MLE theories 27 ) are meaningful? 
Which of them are not? Are they all appropriately operationalized? 

7. Which other indicators would you suggest with regard to this criterion? 

[Questions 6 and 7 are repeated for the remaining four criteria.] 

8. What do you think about the structure of the framework, in particular with regard to the three columns (criteria 
of quality, sub-criteria, and indicators)? 

9. In your view, is the framework clear from the way it is presented? Is it applicable? 

10. What kind of (other) documents and tools would an evaluator need in order to apply the frame to actual MLE 
activities carried out in elementary schools? 


26 The draft can be found at http://qualityMLE.wordpress.com. 

27 In the draft list of indicators I showed the experts, the criteria were differently ordered from the final version of the framework pre¬ 
sented here. 
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Annex 2: 

Protocol for the interviews with elementary school teachers experienced in teaching MLE 
Goals of the interview 

a) Explore which features are perceived as essential with regard to good MLE practice in elementary schools; 

b) verify the adequacy of the framework with regard to the actual situation of MLE in Italian elementary schools; 

c) verify the usefulness and the possible use of such a tool. 

Script for the interviewer 

Part 1 (before showins the draft) 2 * 

1. Can you think of an experience of MLE in elementary schools that you would consider a good practice? If so, 
could you please tell me about it? 

2. In your opinion, what are the elements that contribute to the quality of MLE in elementary schools? 

3. Are these elements feasible in today’s schools? 

4. What are obstacles to the implementation of good MLE practices in elementary schools? 

Part 2 (after showins and explainins the draft) 

1. What do you think about this framework? 

2. What do you think about the five general areas that compose the framework? What other indicators would you 
suggest to add? 

3. Are there items that you think could be expressed better or need further explanation? 

4. What concepts exist for evaluating the quality of individual activities carried out by the teacher during a MLE 
unit? 

5. In your opinion, what means originality in MLE? Is it a criterion of quality? 

6. Are the indicators appropriate with regard to the situation of today’s schools? 

7. In your view, could this tool be of help for teachers or would it cause problems for them? 

8. In your view, is the framework clear from the way it is presented? Is it applicable? 

9. Can such a frame of indicators be useful? What could be the added value of such a framework? 


28 The draft can be found at http://qualityMLE.wordpress.com. 
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